Image-pickup apparatus and method of detecting defective pixel thereof

ABSTRACT

An image pickup apparatus which can detect, when pixels have a structure in which part of electrical construction is shared therebetween, a defective pixel by taking into account a high possibility of the other pixels sharing the part of electrical construction becoming defective pixels, thereby making it possible to obtain an excellent image. A ROM stores in advance position information on each defective pixel. A defective pixel-detecting section detects a new defective pixel on which position information is not stored by the storage unit, from the pixels forming each pixel group, by performing one of different types of defective pixel detection processing. A system controller causes the defective pixel-detecting section to execute one of the different types of detection processing, according to the number of defective pixels which are included in each pixel group and on which the position information is stored in the storage unit.

TECHNICAL FIELD

The present invention relates to an image pickup apparatus including animage pickup device, such as a CCD image sensor and a CMOS image sensor,and a method of detecting a defective pixel of the image pickupapparatus.

BACKGROUND ART

In recent years, an image pickup apparatus tends to increase in thenumber of pixels. Further, there is an increasing demand for high-speedreading of image data as the number of cells is increased by reductionof cell size or so as to adapt to a moving image and the like.

To meet the demand, there has been proposed an image pickup apparatus inwhich a plurality of pixels (photoelectric conversion elements) adjacentto each other share part of electrical construction therebetween (seee.g. PTL (Patent Literature) 1). For example, a floating diffusion unit(FD) (211), a MOS amplifier (212), a selection switch (213) and a resetswitch (210) are shared between photo diodes (202 to 205) of therespective pixels and transfer switches (206 to 209) (see FIG. 3).

Here, in an image pickup device used in the image pickup apparatus,defective pixels are sometimes generated e.g. in a manufacturingprocess. Outputs from the defective pixels are different in output levelfrom other normal pixels, causing image quality degradation.

On the other hand, there has been proposed a technique of correctingdefective pixels of the image pickup device, information on which isstored in advance, checking the image pickup device as occasion demandsto thereby extract defective pixels different in pixel output from theother pixels and store information thereon, and correcting the newdefective pixels thus extracted as well (see PTL 2).

Further, there has been proposed another technique of correctingdefective pixels in real time, by extracting defective pixels based on adifference in signal level between a predetermined pixel of an imagepicked up and pixels of the same color adjacent to the predeterminedpixel, and a difference in signal level between pixels of the same colorof a different kind existing around the predetermined pixel (see PTL 3).

Furthermore, there has been proposed still another technique in whichwhen storing the address of a defective pixel, there are stored not onlythe address of the defective pixel but also a bit of data fordetermining whether a plurality of defective pixels occur in successionthereto, whereby as for defective pixels occurring in succession, onlythe address of a leading one of the defective pixels is stored (see PTL4).

Further, a phenomenon is known in which the increase in number of thepixels of the image pickup apparatus produces pixels suffering from aflickering defect in which when the image pickup device is repeatedlyread out under the same conditions (the same temperature, the sameaccumulation time period, and light blocking), the dark charge level ofan image signal read out undergoes extreme rises and falls (see PTL 5).

CITATION LIST Patent Literature

-   -   PTL 1: Japanese Patent Laid-Open Publication No. H10-256521    -   PTL 2: Japanese Patent Laid-Open Publication No. 2002-125154    -   PTL 3: Japanese Patent Laid-Open Publication No. H06-30425    -   PTL 4: Japanese Patent Laid-Open Publication No. 2008-278394    -   PTL 5: Japanese Patent Laid-Open Publication No. 2003-037781

SUMMARY OF INVENTION Technical Problem

However, the above-described conventional image pickup apparatus suffersfrom the following problems: There is a case where due to the structureof the image pickup device, noise charge is sometimes generated in thefloating diffusion unit, and the noise charge is applied to fourvertically-consecutive pixels that share part of electrical constructiontherebetween. In this case, all of a plurality of adjacent pixelssometimes appear as pixels suffering from the flickering defectdescribed in PTL 5. Such a flickering defect does not always occurperiodically, but depending on the number of times of reading the imagepickup device, the flickering defect occurs with a high frequency e.g.once in several times of reading or with a low frequency e.g. once inseveral years. Further, the flickering defect does not depend on thetemperature or the accumulation time period.

Further, in detecting a defective pixel, as described in PTL 2, whensignal correction is performed by simply using the signals of adjacentpixels, there is a possibility that pixels to be used for correction arenot properly extracted.

Further, as described in PTL 4, if it is determined that all the pixelsthat share the part of electrical construction therebetween aredefective pixels, assuming that the defective pixels suffer from theflickering defect, even when a signal of a read one of the pixels showsa correct output, the signal of the pixel is corrected or replaced byoutput signals of pixels around the pixel. This can result in thedegraded image quality.

In the case of such a structure of the pixels sharing part of electricalconstruction, it is necessary to properly detect defective pixels causedby the above-mentioned factors.

The present invention has been made in view of these circumstances, andprovides an image pickup apparatus which is, when pixels have astructure in which part of electrical construction is shared betweenthem, capable of detecting a defective pixel by taking into account ahigh possibility of the other pixels sharing the part of electricalconstruction becoming defective pixels, thereby making it possible toobtain an excellent image. Further, the present invention provides amethod of detecting defective pixels.

Solution to Problem

Accordingly, in a first aspect of the present invention, there isprovided an image pickup apparatus including a solid image pickup devicefor converting an optical image formed by a plurality of pixel groupsarranged two-dimensionally to an electric signal and outputting theelectric signal, each pixel group being formed by a plurality of pixelshaving a pixel structure in which two or more pixels share part ofelectrical construction therebetween, comprising a storage unitconfigured to store position information on each of defective pixels inadvance, a defective pixel-detecting unit configured to detect a newdefective pixel on which position information is not stored in thestorage unit, out of the plurality of pixels forming each pixel group,by one of a plurality of different types of defective pixel detectionprocessing, and a control unit configured to cause the defectivepixel-detecting unit to execute the one of the plurality of types ofdefective pixel detection processing, according to the number ofdefective pixels which are included in each pixel group and on which theposition information is stored in the storage unit.

Accordingly, in a second aspect of the present invention, there isprovided an image pickup apparatus including a solid image pickup devicefor converting an optical image formed by a plurality of pixel groupsarranged two-dimensionally to an electric signal and outputting theelectric signal, each pixel group being formed by a plurality of pixelshaving a pixel structure in which two or more pixels share part ofelectrical construction therebetween, comprising a storage unitconfigured to store position information on each of defective pixels inadvance, a defective pixel-detecting unit configured to detect a newdefective pixel on which position information is not stored in thestorage unit, out of the plurality of pixels forming each pixel group,by a plurality of different types of defective pixel detectionprocessing, and a control unit configured to add respective results ofthe plurality of different types of defective pixel detection processingby weighted averaging dependent on the number of defective pixels whichare included in each pixel group and on which the position informationis stored in the storage unit, and determine based on a result of theaddition whether or not a detection target pixel is the new defectivepixel.

Accordingly, in a third aspect of the present invention, there isprovided a method of detecting a defective pixel for an image pickupapparatus including a solid image pickup device for converting anoptical image formed by a plurality of pixel groups arrangedtwo-dimensionally to an electric signal and outputting the electricsignal, each pixel group being formed by a plurality of pixels having apixel structure in which two or more pixels share part of electricalconstruction therebetween, comprising detecting a new defective pixel onwhich position information is not stored in a storage unit, out of theplurality of pixels forming each pixel group, by one of a plurality ofdifferent types of defective pixel detection processing, and causing thedetecting to be executed by the one of the plurality of types ofdefective pixel detection processing, according to the number ofdefective pixels which are included in each pixel group and on which theposition information is stored in the storage unit.

Accordingly, in a fourth aspect of the present invention, there isprovided a method of detecting a defective pixel for an image pickupapparatus including a solid image pickup device for converting anoptical image formed by a plurality of pixel groups arrangedtwo-dimensionally to an electric signal and outputting the electricsignal, each pixel group being formed by a plurality of pixels having apixel structure in which two or more pixels share part of electricalconstruction therebetween, comprising detecting a new defective pixel onwhich position information is not stored in a storage unit, out of theplurality of pixels forming each pixel group, by a plurality ofdifferent types of defective pixel detection processing, and addingrespective results of the plurality of different types of defectivepixel detection processing by weighted averaging dependent on the numberof defective pixels which are included in each pixel group and on whichthe position information is stored in the storage unit, and determiningbased on a result of the addition whether or not a detection targetpixel is the new defective pixel.

Advantageous Effects of Invention

According to the present invention, it is possible, when pixels have astructure in which part of electrical construction is shared betweenthem, to detect a defective pixel by taking into account a highpossibility of the other pixels sharing the part of electricalconstruction becoming defective pixels, and thereby obtain an excellentimage.

The features and advantages of the invention will become more apparentfrom the following detailed description taken in conjunction with theaccompanying drawings.

BRIEF DESCRIPTION OF DRAWINGS

[FIG. 1] A schematic block diagram of an image pickup apparatusaccording to a first embodiment.

[FIG. 2] A view of an example of a pixel array included in an imagepickup device.

[FIG. 3] A view of an equivalent circuit of a pixel group having a pixelstructure in which four pixels arranged adjacent to each other in avertical direction share a floating diffusion unit.

[FIG. 4] A flowchart of a defective pixel detection process in a casewhere the image pickup device comprises pixel groups each having a pixelstructure in which pixels share part of electrical constructiontherebetween.

[FIG. 5] A view useful in explaining median filtering in fourdirections.

[FIG. 6] A view useful in explaining median filtering in a case whereone of pixels B, B, and G sharing the floating diffusion unit with apixel G32 is a defective pixel the address of which is stored in advancein a ROM.

[FIG. 7] A view useful in explaining median filtering in a case wheretwo of the pixels B, B, and G sharing the floating diffusion unit withthe pixel G32 are defective pixels the addresses of which are stored inadvance in the ROM.

[FIG. 8] A view useful in explaining median filtering in a case where hepixels B, B, and G sharing the floating diffusion unit with the pixelG32 are defective pixels the addresses of which are stored in advance inthe ROM.

[FIG. 9] A view useful in explaining median filtering in four directionsperformed by an image pick-up apparatus according to a secondembodiment, as first defective pixel detection processing.

[FIG. 10] A view useful in explaining median filtering in threedirections as second defective pixel detection processing.

[FIG. 11] A graph showing an example of setting of a coefficient K.

[FIG. 12] A flowchart of a defective pixel detection process in a casewhere the image pickup device comprises pixel groups each having a pixelstructure in which pixels share part of electrical constructiontherebetween.

DESCRIPTION OF EMBODIMENTS

The present invention will now be described in detail below withreference to the accompanying drawings showing embodiments thereof.

A description will be given of embodiments of an image pickup apparatusand a method of detecting a defective pixel of the image pickupapparatus according to the present invention with reference to thedrawings.

FIG. 1 is a schematic block diagram of the image pickup apparatusaccording to a first embodiment. The image pickup apparatus 50 accordingto the present embodiment includes an optical system 1 comprisinglenses, not shown, and a diaphragm, not shown, a mechanical shutter 2and an image pickup device 3.

In the present embodiment, a CMOS (complementary metal oxidesemiconductor) area sensor arranged in the Bayer array is used as theimage pickup device 3. FIG. 2 is a view of an example of a pixel arrayincluded in the image pickup device 3. The image pickup device (solidimage pickup device) 3 includes a plurality of pixel groups arrangedtwo-dimensionally in the vertical and horizontal directions. In FIG. 2,symbols R (red), G (green), and B (blue) represent colors received afterpassing through color filters of R, G and B (red, green, and blue)colors formed on respective pixels of the image pickup device 3.

Further, in the image pickup device 3 according to the presentembodiment, a plurality of (not smaller than 2) pixels arranged adjacentto each other in one direction (in the vertical direction, in theillustrated example) share part of electrical construction therebetween.FIG. 3 is a view of an equivalent circuit of a pixel group having apixel structure in which four pixels arranged adjacent to each other inthe vertical direction share a floating diffusion unit (FD) 211. In thispixel structure, the floating diffusion unit 211, a MOS amplifier 212, aselection switch 213 and a reset switch 210 are shared between photodiodes (photoelectric conversion elements) 202 to 205 of the respectivepixels and transfer switches 206 to 209. The floating diffusion unit 211(holding unit) is provided for common use by the pixels of a pixel grouphaving a pixel structure in which the pixels arranged adjacent to eachother in the vertical direction share part of electrical constructiontherebetween, and temporarily stores electric charge generated in thephoto diode of each pixel.

An object image (optical image) formed on the pixel groups of the imagepickup device 3 is converted by the image pickup device 3 to an electricsignal. Then, the electric signal (image signal) delivered from theimage pickup device 3 is input to a CDS (co-related double sampling)circuit 4.

The CDS circuit 4 performs analog signal processing on the input imagesignal. An A/D converter 5 converts an analog signal from the CDScircuit 4 to a digital signal. A timing signal-generating circuit 6generates signals for causing the image pickup device 3, the CDS circuit4 and the A/D converter 5 to operate. A drive circuit 7 drives theoptical system 1, the mechanical shutter 2 and the image pickup device3.

The image signal that has been converted to the digital signal by theA/D converter 5 is input to a defective pixel-detecting & correctingunit 8. The defective pixel-detecting & correcting unit 8 comprises adefective pixel-detecting section 81 and a defective pixel-correctingsection 82. Upon receipt of the digital image signal, the defectivepixel-detecting section 81 (defective pixel detecting unit) detects adefective pixel from a plurality of pixels forming each of a pluralityof pixel groups, and outputs information on the defective pixel asdefective pixel information to the defective pixel-correcting section82. The defective pixel-detecting section 81 mainly detects defectivepixels, such as a flickering defective pixel, position information ofwhich is not stored in a ROM 16, referred to hereinafter.

The defective pixel-correcting section 82 performs defective pixelcorrection processing on the input image signal according to thedefective pixel information output from the defective pixel-detectingsection 81. Detailed description of the defective pixel-detecting &correcting unit 8 will be given hereinafter.

An output from the defective pixel-detecting & correcting unit 8 isinput to a signal processing circuit 9. The signal processing circuit 9performs required digital signal processing on the output from thedefective pixel-detecting & correcting unit 8, and then outputs theprocessed image signal as image data.

An image memory 10 is an area for temporarily storing a video signalbeing subjected to the digital signal processing by the signalprocessing circuit 9, or storing image data. A storage medium 11 is amemory card or the like which is removably mounted on the image pickupapparatus 50.

A recording circuit 12 records image data obtained by the digital signalprocessing by the signal processing circuit 9 in the storage medium 11.An image display device 13 displays the image data obtained by thedigital signal processing by the signal processing circuit 9 via adisplay circuit 14.

A system controller 15 controls the overall operation of the imagepickup apparatus 50. In the present embodiment, the system controller 15is formed e.g. by a CPU. The ROM 16 (storage unit) stores a programdescribing a method of controlling the image pickup apparatus 50,executed by the system controller 15, control data, such as parametersand tables, used during execution of the program, and correction data,such as addresses (position information) of defective pixels included inthe pixel groups of the image pickup device 3.

In the present embodiment, out of the above-mentioned correction data,correction data items associated with respective sensors checked andsent by a manufacturing plant that manufactures the image pickup device3 are stored as defective pixel-correcting data in the ROM 16. But, themethod of acquiring the defective pixel-correcting data is not limitedto this. For example, new information on defective pixels may be createdby the defective pixel-detecting section 81 and stored in the ROM 16after the image pickup device 3 has been incorporated in the imagepickup apparatus 50.

The program, the control data, the correction data, such as theaddresses of the defective pixels, and so forth stored in the ROM 16 aretransferred to a RAM 17 for temporary storage therein. Further,additional programs and data are transferred from the ROM 16 to the RAM17, as required. Further, the system controller 15 sometimes directlyreads out data from the ROM 16, for use.

Next, a description will be given of a shooting operation performed byusing the mechanical shutter 2 in the image pickup apparatus 50configured as described above. First, the optical system 1 drives thediaphragm and the lenses via the drive circuit 7 according to a controlsignal from the system controller 15, to thereby form an object imageset to appropriate brightness, on the image pickup device 3.

The mechanical shutter 2 is driven by the drive circuit 7 by a controlsignal from the system controller 15 such that light to the image pickupdevice 3 is blocked in accordance with the operation of the image pickupdevice 3, to thereby expose the image pickup device 3 to the lightduring required exposure time. At this time, when the image pickupdevice 3 has an electronic shutter function, the required exposure timemay be ensured by using the image pickup device 3 in combination withthe mechanical shutter 2.

The image pickup device 3 is driven by drive pulses based on operationpulses generated by the timing signal-generating circuit 6 controlled bythe system controller 15, and converts an object image to an electricsignal by photoelectric conversion, to thereby output the electricsignal as an analog image signal.

The CDS circuit 4 removes clock-synchronous noise from the electricsignal (analog image signal) delivered from the image pickup device 3using the operation pulses generated by the timing signal-generatingcircuit 6 controlled by the system controller 15. Further, the analogimage signal from which the clock-synchronous noise has been removed isconverted to a digital image signal by the A/D converter 5.

The digital image signal is once stored in the image memory 10 under thecontrol of the system controller 15, and is then input to the defectivepixel-detecting & correcting unit 8. The defective pixel-detectingsection 81 detects a defective pixel from the input digital imagesignal, and outputs the result of the detection as information of thedefective pixel to the defective pixel-correcting section 82.

The defective pixel-correcting section 82 performs defective pixelcorrection processing on the input digital image signal based ondefective pixel correction data, such as the addresses of defectivepixels stored in the ROM 16, and the defective pixel information outputfrom the defective pixel-detecting section 81.

The signal processing circuit 9 performs image processing, such as colorconversion, white balance, and gamma correction, resolution conversionprocessing, image compression processing, and so forth on the digitalimage signal having been subjected to the defective pixel detection &correction processing by the defective pixel-detecting & correcting unit8.

The image data stored in the image memory 10 is converted to datasuitable for the storage medium 11 (e.g. file system data having ahierarchical structure) by the recording circuit 12. This convertedimage data is recorded in the storage medium 11. Further, after beingsubjected to the resolution conversion processing in the signalprocessing circuit 9, the converted image data is converted to a signalsuitable for the image display device 13 (e.g. an NTSC analog signal) bythe display circuit 14, and is displayed on the image display device 13.

The signal processing circuit 9 may directly deliver the digital imagesignal to the image memory 10 or the recording circuit 12 as image datawithout subjecting the digital image signal to the above-described imageprocessing by the control signals from the system controller 15.

Next, a detailed description will be given of defective pixel detectionprocessing performed in a case where the image pickup device 3 comprisespixel groups each having a pixel structure in which pixels arrangedadjacent to each other in the vertical direction share part ofelectrical construction therebetween. In the illustrated example, pixelsB22, G32, B42, and G52 of the image pickup device 3 form a pixel grouphaving a pixel structure in which the four pixels share the floatingdiffusion unit 211. Now, a description will be given of a case where adetection target pixel being subjected to defective pixel detection isthe pixel G32, by way of example (see FIG. 5).

Further, various methods for detecting defective pixels whenever animage is picked up have been proposed, as disclosed in Patent Literature3. In the present embodiment, however, a description will be given ofmedian filtering for extracting a central value of data of images havingthe same color in a range of 5×5 pixels in the vertical and horizontaldirections, by way of example.

FIG. 4 is a flowchart of a defective pixel detection process in the casewhere the image pickup device 3 comprises pixel groups each having thepixel structure in which the pixels arranged adjacent to each other inthe vertical direction share part of electrical constructiontherebetween. A program for this defective pixel detection process isstored in the ROM 16, as described above, and is executed by the systemcontroller 15 (CPU: control unit). In the illustrated example, thedetection target pixel is the pixel G32.

First, the system controller 15 reads out picked-up image data from theimage memory 10 to the defective pixel-detecting & correcting unit 8(step S1). The defective pixel-detecting section 81 refers to thedefective pixel correction data stored in the ROM 16, in advance, tothereby determine whether or not the detection target pixel G32 is adefective pixel (step S2).

If the detection target pixel G32 is a defective pixel the address(position information) of which is stored in advance in the ROM 16(storage unit), the detection target pixel G32 is required to besubjected to the defective pixel correction processing, and hence thesystem controller 15 causes the pixel G32 to be subjected to thedefective pixel correction processing by the defective pixel-correctingsection 82 (step S9). In the defective pixel correction processing,correction of the defective pixel is performed using a value obtained bypre-interpolation or interpolation from adjacent same color pixels.After that, the system controller 15 terminates the present process.

On the other hand, in the step S2, if the detection target pixel G32 isnot a defective pixel the address of which is stored in advance in theROM 16, the system controller 15 determines using the defectivepixel-detecting section 81 whether or not any of the pixels B22, B42,and G52 sharing the floating diffusion unit 211 with the detectiontarget pixel G32 is a defective pixel the address of which is stored inadvance in the ROM 16 (step S3).

If it is determined in the step S3 that none of the pixels B22, B42, andG52 sharing the floating diffusion unit 211 with the detection targetpixel G32 are defective pixels the addresses of which are stored inadvance in the ROM 16, the system controller 15 causes the defectivepixel-detecting section 81 to execute first defective pixel detectionprocessing (step S5). More specifically, as shown in FIG. 5, medianfiltering in four directions is performed on the detection target pixelG32. FIG. 5 shows median filtering in the four directions.

After completion of the median filtering in the four directions, thedefective pixel-detecting section 81 determines whether or not thedetection target pixel G32 is a defective pixel (step S7). In thepresent embodiment, if the detection target pixel G32 has an outputvalue larger than a predetermined threshold value, it is determined thatthe detection target pixel G32 is a defective pixel. Here, thepredetermined threshold value used in the above determination is set asdesired to a level at which the detection target pixel G32 cause aproblem to the image compared with the other pixels.

Assuming that the predetermined threshold value for use in thedetermination of whether or not the detection target pixel G32 is adefective pixel is represented by TH, it is determined that thedetection target pixel G32 is a defective pixel e.g. if the followingnumerical expressions (1) to (4) are all satisfied.

G32−median(G30, G32, G34)>TH   (1)

G32−median(G21, G32, G43)>TH   (2)

G32−median(G23, G32, G41)>TH   (3)

G32−median(G12, G32, G52)>TH   (4)

Image data obtained by median filtering is image data smoothed byremoving high-frequency components therefrom so as to eliminate theadverse influence of the defective pixel. Therefore, in the step S5, byperforming subtraction processing for subtracting the smoothed imagedata from data of the detection target pixel, it is possible to obtain aprominent output value which is a noise component.

Note that when a defective pixel the address of which is stored inadvance in the ROM 16 exists in a same color pixel range within whichmedian filtering is performed, the defective pixel is handled as one tobe excluded from calculation, and median filtering is performed usingdata of the pixels other than the defective pixel. This enablesdefective pixel detection to be performed with higher accuracy.

If it is determined in the step S7 that the detection target pixel G32is a defective pixel, the system controller 15 performs the defectivepixel correction processing using the defective pixel-correcting section82 in the step S9, and then terminates the present process.

Next, a description will be given of a case where it is determined inthe step S3 that one of the pixels B22, B42, and G52 sharing thefloating diffusion unit 211 with the pixel G32 is a defective pixel theaddress of which is stored in advance in the ROM 16. FIG. 6 is a viewuseful in explaining median filtering in the case where one of thepixels B22, B42, and G52 sharing the floating diffusion unit 211 withthe pixel G32 is a defective pixel the address of which is stored inadvance in the ROM 16.

For example, when only the pixel B22 of the pixels B22, B42, and G52sharing the floating diffusion unit 211 with the pixel G32 is adefective pixel, the system controller 15 determines that the pixel B22is only one defective pixel. In other words, the system controller 15determines whether or not the number of the defective pixels is equal to1 (step S4), and if the number of the defective pixels is equal to 1,the system controller 15 executes the above-described step S5 et seq.That is, similarly to the above-described case where none of the pixelssharing the floating diffusion unit 211 with the pixel G32 are defectivepixels the addresses of which are stored in advance in the ROM 16, inthe step S5, the system controller 15 performs median filtering in thefour directions as the defective pixel detection processing, as shown inFIG. 6, and determines whether or not the pixel G32 is a defectivepixel, using the numerical expressions (1) to (4). Then, if it isdetermined in the step S7 that the pixel G32 is a defective pixel, thesystem controller 15 performs the defective pixel correction processingin the step S9.

Next, a description will be given of a case where two of the pixels B22,B42, and G52 sharing the floating diffusion unit 211 with the pixel G32are defective pixels the addresses of which are stored in advance in theROM 16. FIG. 7 is a view useful in explaining median filtering in thecase where two of the pixels B22, B42, and G52 sharing the floatingdiffusion unit 211 with the pixel G32 are defective pixels the addressesof which are stored in advance in the ROM 16.

For example, when the pixels B22 and B42 of the three pixels B22, B42,and G52 sharing the floating diffusion unit 211 with the pixel G32 aredefective pixels, since two of the four pixels sharing the floatingdiffusion unit 211 in the pixel structure are defective pixels, there isa high possibility that the floating diffusion unit 211 suffers fromoccurrence of a phenomenon that noise charge is sometimes caused by astructural factor and is applied in succession to thevertically-arranged four pixels which share the floating diffusion unit211 between them, whereby it is determined that the pixel G52 as wellhas a high possibility of being a defective pixel.

Therefore, if it is determined in the step S4 that the number of thedefective pixels is not equal to 1, i.e. not smaller than 2, the systemcontroller 15 causes the defective pixel-detecting section 81 to executesecond defective pixel detection processing different from the firstdefective pixel detection processing in the step S5. More specifically,as shown in FIG. 7, the system controller 15 performs median filteringin three directions except for the direction of arrangement of thepixels (B22, B42, and G52) sharing the floating diffusion unit 211 withthe detection target pixel G32 (step S6). After execution of medianfiltering in the three directions, the defective pixel-detecting section81 determines whether or not the detection target pixel G32 is adefective pixel (step S8). Note that this processing may be replaced byprocessing in which median filtering is performed in the four directionsbut the defective pixel-detecting section 81 does not refer to a resultof median filtering in the direction of arrangement of the pixelssharing the floating diffusion unit 211 with the detection target pixelG32.

In other words, when it is determined whether or not the detectiontarget pixel G32 is a defective pixel, the determination is performed byexecuting only the numerical expressions (1) to (3). Alternately,although calculation using the numerical expression (4) is alsoperformed, the result of the calculation is not referred to. If thenumerical expressions (1) to (3) are all satisfied, it is determinedthat the detection target pixel G32 is a defective pixel. If it isdetermined in the step S8 that the detection target pixel G32 is adefective pixel, in the step S9, the system controller 15 performs thedefective pixel correction processing using the defectivepixel-correcting section 82.

Next, a description will be given of a case where all the pixels B22,B42, and G52 except for the detection target pixel G32, sharing thefloating diffusion unit 211 with the pixel G32 are defective pixels theaddresses of which are stored in advance in the ROM 16. FIG. 8 is a viewuseful in explaining median filtering in the case where the pixels B22,B42, and G52 sharing the floating diffusion unit 211 with the pixel 32are defective pixels the addresses of which are stored in advance in theROM 16.

As shown in FIG. 8, median filtering is performed in the threedirections, and the determination of whether or not all the pixels B22,B42, and G52 are defective pixels is performed by executing only thenumerical expressions (1) to (3). That is, when it is determined in thestep S8 that all the pixels B22, B42, and G52 are defective pixels, thesystem controller 15 performs the defective pixel correction processingusing the defective pixel-correcting section 82 in the step S9.

As described heretofore, according to the image pickup apparatus of thefirst embodiment, it is possible to perform defective pixel detectionand obtain an excellent image, by taking into account a higherpossibility that in the case of the image pickup device 3 comprisingpixel groups each having a pixel structure in which pixels in each pixelgroup share part of electrical construction therebetween, if a pixel inthe pixel group is a defective pixel, pixels in the pixel group otherthan the pixel are also defective pixels.

Although in the above-described embodiment, the description has beengiven of the case where the detection target pixel being subjected todefective pixel detection is a G pixel, the same processing as in theabove-described case is performed also in cases where the detectiontarget pixel is a B pixel or an R pixel, and hence description thereofis omitted.

Further, in the above-described embodiment, it is shown that in the caseof a pixel structure in which each pixel group formed by four pixelsarranged in the vertical direction share an associated floatingdiffusion unit 211, when two or more pixels including a pixel of thepixel group sharing the associated floating diffusion unit 211 aredefective pixels information on which is stored in advance, medianfiltering in the vertical direction is ignored. However, the presentinvention is effective not only in the case where the pixel group has apixel structure in which pixels arranged in one direction share thefloating diffusion unit 211 but also in a case where the pixel group hasa pixel structure in which pixels arranged in two directions share thefloating diffusion unit 211. That is, when two or more pixels includinga pixel of the pixel group sharing the associated floating diffusionunit 211 are defective pixels information on which is stored in advance,it is only required to ignore median filtering in the direction ofarrangement of pixels having the same color.

Further, similarly, the present invention can be applied to a case wherethe number of pixels of a pixel group sharing the floating diffusionunit 211 is not smaller than four, by switching between two or moretypes of defective pixel detection processing.

Further, although in the present embodiment, median filtering isemployed as the defective pixel detection processing, this is notlimitative, but difference-based processing may be employed which isperformed based on the difference between signal levels of apredetermined pixel and a same color pixel adjacent to the predeterminedpixel. The same applies to the following embodiment.

Next, a description will be given of a second embodiment of the presentinvention. In the second embodiment, detection of a defective pixel isperformed by weighted averaging in which results of two types ofdefective pixel detection processing are respectively weighted andaveraged. FIG. 9 is a view useful in explaining median filtering in thefour directions according to the second embodiment, as first defectivepixel detection processing. FIG. 10 is a view useful in explainingmedian filtering in the three directions as second defective pixeldetection processing. The configuration of the image pickup apparatus isthe same as that of the above-described image pickup apparatus accordingto the first embodiment, and hence description thereof is omitted byusing the same reference numerals.

Similarly to the first embodiment, the image pickup device 3 has a pixelstructure in which the pixels B22, G32, B42, and G52 of a pixel groupshare an associated floating diffusion unit 211, and the detectiontarget pixel being subjected to defective pixel detection is the pixelG32, by way of example.

Referring to FIG. 9, median filtering is performed in the fourdirections, and processing for subtracting image data smoothed accordingto a numerical expression (5) from data of the detection target pixel,whereby a prominent output value A which is a noise component isobtained.

G32−median(G12, G21, G23, G30, G32, G34, G41, G43, G52)   (5)

Further, as shown in FIG. 10, the median filtering is performed in threedirections except for the direction of arrangement of pixels sharing thefloating diffusion unit 211 with the detection target pixel G32, andprocessing for subtracting image data smoothed according to a numericalexpression (6) from data of the detection target pixel, whereby aprominent output value B which is a noise component is obtained.

G32−median(G21, G23, G30, G32, G34, G41, G43)   (6)

Here, assuming that a predetermined threshold value used in thedetermination of a defective pixel is represented by TH, a sum value(weighted sum value) MIX obtained by weighted averaging of the outputvalue A and the output value B is calculated e.g. by the followingequation (7):

MIX=A×K+B×(1−K)   (7)

wherein a coefficient K represents a value from 0 to 1, which isdetermined by the number of defective pixels the addresses of which arestored in advance in the ROM 16 existing in a pixel group sharing thefloating diffusion unit 211.

FIG. 11 is a graph showing an example of the setting of the coefficientK. For example, when there is no defective pixel the address of which isstored in advance in the ROM 16, in the pixel group sharing the floatingdiffusion unit 211, K=1 is set, and an output value obtained by themedian filtering in all the four directions is employed.

Further, when all the pixels B22, B42, and G52 except for the detectiontarget pixel G32, which share the floating diffusion unit 211 with thepixel G32, are defective pixels the addresses of which are stored inadvance in the ROM 16, K=0 is set, and an output value obtained bymedian filtering in the three directions is employed.

Furthermore, when one of the pixels B22, B42, and G52 sharing thefloating diffusion unit 211 with the pixel G32 is a defective pixel theaddress of which is stored in advance in the ROM 16, there is a highpossibility that it is only one defective pixel, and hence thecoefficient K is set to a value close to 1, and an output value isemployed which is obtained by attaching importance to the result of themedian filtering in all the four directions. In the illustrated example,the coefficient K is set to 0.6.

On the other hand, when two of the pixels B22, B42, and G52 sharing thefloating diffusion unit 211 with the pixel G32 are defective pixels theaddresses of which are stored in advance in the ROM 16, there is a highpossibility that noise charge is sometimes caused in the floatingdiffusion unit 211 due to a structural factor, and the noise charge isapplied in succession to the vertically-arranged four pixels sharing thefloating diffusion unit 211. Therefore, the coefficient K is set to avalue close to 0, and an output value is employed which is obtained byattaching importance to the result of median filtering in threedirections. In the illustrated example, the coefficient K is set to 0.3.As described hereinabove, in the present embodiment, the defective pixelis detected based on the result of addition using weighted averaging inwhich respective results of the two types of defective pixel detectionprocessing are weighted.

Similarly to the first embodiment, assuming that a predetermined value(threshold value) used in the determination of a defective pixel isrepresented by TH, when the relationship between the predetermined valueTH and the weighted sum value MIX obtained as above, expressed by thefollowing numerical expression (8), is satisfied, it is determined thatthe detection target pixel G32 is a defective pixel.

MIX>TH   (8)

Here, if it is determined that there is a defective pixel the address ofwhich is stored in advance in the ROM 16 in a same color pixel rangewithin which median filtering is performed, the defective pixel ishandled as one to be excluded from calculation (excluded fromdetermination), and median filtering is performed using data of thepixels other than the defective pixel.

Further, although in the above-described embodiment, the number ofpixels in each pixel group sharing the floating diffusion unit 211 withthe detection target pixel being subjected to defective pixel detectionis three, the present embodiment can also be applied to the case wherethe number of pixels in the pixel group is not smaller than four, as thesecond embodiment, by calculating a weighted sum value using a weightedsum value of results of two or more types of defective pixel detectionprocessing.

FIG. 12 is a flowchart of a defective pixel detection process in a casewhere the image pickup device 3 comprises pixel groups each having apixel structure in which pixels in each pixel group share part ofelectrical construction therebetween. A program for this defective pixeldetection process is stored in the ROM 16, as described above, and isexecuted by the system controller 15 (CPU). Description of stepsidentical to those in the defective pixel detection process shown inFIG. 4 of the first embodiment is omitted by designating the steps usingidentical step numbers.

If it is determined in the step S2 that the detection target pixel G32is not a defective pixel, based on the defective pixel informationstored in advance in the ROM 16, the system controller 15 obtains thenumber of defective pixels, out of the pixels sharing the floatingdiffusion unit 211, the addresses of which are stored in advance in theROM 16 (step S3A).

The system controller 15 causes the defective pixel-detecting section 81to execute the above-described plurality of types of defective pixeldetection processing different from each other. Then, the systemcontroller 15 calculates the weighted sum value MIX by the numericalexpression (7) based on the number of the defective pixels obtained inthe step S3A (step S5A). Based on the result of this calculation(addition), the system controller 15 determines whether or not thedetection target pixel G32 is a defective pixel, i.e. whether or not theweighted sum value MIX of the detection target pixel G32 satisfies thenumerical expression (8) (step S7A).

If the detection target pixel G32 is not a defective pixel, the systemcontroller 15 immediately terminates the present process, whereas if thedetection target pixel G32 is a defective pixel, the system controller15 performs defective pixel correction processing using the defectivepixel-correcting section 82. In this defective pixel correctionprocessing, as described hereinabove, correction of the defective pixelis performed using a value obtained by pre-interpolation orinterpolation from adjacent same color pixels.

As described heretofore, according to the second embodiment, it ispossible to detect a defective pixel by causing the number of defectivepixels, out of the pixels sharing the floating diffusion unit 211,stored in advance in the ROM 16 to be reflected to a further extent.

The present invention is not limited to the above-described embodiments,but can be modified in various manners based on the subject matter ofthe present invention, which should not be excluded from within thescope of the present invention insofar as functions as recited in theappended claims or the functions performed by the construction of eachof the above described embodiments can be achieved.

Further, the present invention can be applied to not only color imagepickup devices of colors of R, G, B, or the like, but also to amonochrome image pickup device.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

REFERENCE SIGNS LIST

-   3 image pickup device-   8 defective pixel-detecting & correcting unit-   15 system controller-   81 defective pixel-detecting section-   82 defective pixel-correcting section-   50 image pickup apparatus-   211 floating diffusion unit

1. An image pickup apparatus including a solid image pickup device forconverting an optical image formed by a plurality of pixel groupsarranged two-dimensionally to an electric signal and outputting theelectric signal, each pixel group being formed by a plurality of pixelshaving a pixel structure in which two or more pixels share part ofelectrical construction therebetween, comprising: a storage unitconfigured to store position information on each of defective pixels inadvance; a defective pixel-detecting unit configured to detect a newdefective pixel on which position information is not stored in saidstorage unit, out of the plurality of pixels forming each pixel group,by one of a plurality of different types of defective pixel detectionprocessing; and a control unit configured to cause said defectivepixel-detecting unit to execute the one of the plurality of types ofdefective pixel detection processing, according to the number ofdefective pixels which are included in each pixel group and on which theposition information is stored in said storage unit.
 2. The image pickupapparatus according to claim 1, wherein each pixel group is formed by aplurality of pixels arranged adjacent to each other in one direction,wherein when the number of the defective pixels on which the positioninformation is stored in said storage unit is not larger than 1, saidcontrol unit causes, out of the plurality of types of defective pixeldetection processing, first defective pixel detection processing to beexecuted in which median filtering is performed in four directionsincluding the one direction, and when the number of the defective pixelson which the position information is stored in said storage unit is notsmaller than 2, said control unit causes, out of the plurality of typesof defective pixel detection processing, second defective pixeldetection processing to be executed in which the median filtering isperformed in three directions except for the one direction, and whereinsaid defective pixel-detecting unit detects the new defective pixel whenvalues obtained by performing the median filtering in the respectivefour and three directions exceeds a predetermined threshold value.
 3. Animage pickup apparatus including a solid image pickup device forconverting an optical image formed by a plurality of pixel groupsarranged two-dimensionally to an electric signal and outputting theelectric signal, each pixel group being formed by a plurality of pixelshaving a pixel structure in which two or more pixels share part ofelectrical construction therebetween, comprising: a storage unitconfigured to store position information on each of defective pixels inadvance; a defective pixel-detecting unit configured to detect a newdefective pixel on which position information is not stored in saidstorage unit, out of the plurality of pixels forming each pixel group,by a plurality of different types of defective pixel detectionprocessing; and a control unit configured to add respective results ofthe plurality of different types of defective pixel detection processingby weighted averaging dependent on the number of defective pixels whichare included in each pixel group and on which the position informationis stored in said storage unit, and determine based on a result of theaddition whether or not a detection target pixel is the new defectivepixel.
 4. The image pickup apparatus according to claim 3, wherein eachpixel group is formed by a plurality of pixels arranged adjacent to eachother in one direction, wherein the plurality of types of defectivepixel detection processing include first defective pixel detectionprocessing in which median filtering is performed in four directionsincluding the one direction, and second defective pixel detectionprocessing in which the median filtering is performed in threedirections except for the one direction.
 5. The image pickup apparatusaccording to claim 1, wherein the plurality of pixels of each pixelgroup share therebetween a holding unit configured to temporarily holdan electric charge generated in each pixel of the pixel group.
 6. Amethod of detecting a defective pixel for an image pickup apparatusincluding a solid image pickup device for converting an optical imageformed by a plurality of pixel groups arranged two-dimensionally to anelectric signal and outputting the electric signal, each pixel groupbeing formed by a plurality of pixels having a pixel structure in whichtwo or more pixels share part of electrical construction therebetween,comprising: detecting a new defective pixel on which positioninformation is not stored in a storage unit, out of the plurality ofpixels forming each pixel group, by one of a plurality of differenttypes of defective pixel detection processing; and causing saiddetecting to be executed by the one of the plurality of types ofdefective pixel detection processing, according to the number ofdefective pixels which are included in each pixel group and on which theposition information is stored in the storage unit.
 7. A method ofdetecting a defective pixel for an image pickup apparatus including asolid image pickup device for converting an optical image formed by aplurality of pixel groups arranged two-dimensionally to an electricsignal and outputting the electric signal, each pixel group being formedby a plurality of pixels having a pixel structure in which two or morepixels share part of electrical construction therebetween, comprising:detecting a new defective pixel on which position information is notstored in a storage unit, out of the plurality of pixels forming eachpixel group, by a plurality of different types of defective pixeldetection processing; and adding respective results of the plurality ofdifferent types of defective pixel detection processing by weightedaveraging dependent on the number of defective pixels which are includedin each pixel group and on which the position information is stored inthe storage unit, and determining based on a result of the additionwhether or not a detection target pixel is the new defective pixel. 8.The image pickup apparatus according to claim 2, wherein the pluralityof pixels of each pixel group share therebetween a holding unitconfigured to temporarily hold an electric charge generated in eachpixel of the pixel group.
 9. The image pickup apparatus according toclaim 3, wherein the plurality of pixels of each pixel group sharetherebetween a holding unit configured to temporarily hold an electriccharge generated in each pixel of the pixel group.
 10. The image pickupapparatus according to claim 4, wherein the plurality of pixels of eachpixel group share therebetween a holding unit configured to temporarilyhold an electric charge generated in each pixel of the pixel group.